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ABSTRACT 

In a series of four interrelated reports. Rand work 
to date for the Air Force on the development of methodologies for 
designing programs of instruction is reported, Thie report, the third 
of the series, presents and discusses the use of a questionnaire that 
help curriculum designers to analyze a course of study to gauge its 
requirements for communication media, personnel, and time. Use of the 
questionnaire fits as a middle step in the larger sequence of MODIA 
(A Method of Designing Instructional Alternatives) . The questionnaire 
causes the designer to subdivide each course into basic "learning 
events'* and proceed through the branching questionnaire for each- 
(Author/RH) 
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PREFACE 



This is one of four interrelated reports describing Rand work for the Air Force 
to date on the development of methodologies for designing programs of instruction. 
The reports in the series are: 

R-1018-PR, An Overview of MODIA: A Method of Designing Instructional 

Alternatives for Air Force Training, Polly Carpenter. 
R-1019-PR, The MODIA Decision Process for Developing Strategies of Air 

Force Instruction, Polly Carpenter and Barbara Horner. 
R-1020-PR, The MODIA Questionnaire for Curriculum Analysis^ Rudy 

Bretz. 

R-1021-PR, MODIA Applied in the Design and Cost Analysis of an Innova- 
tive Air Force CoursCj Robert L. Petruschell and Polly Carpenter. 

The first of these provides an overview of the methodologies being developed; the 
second and third describe some of the major analytical tools used to provide inputs 
to the design process; and the last sets forth the results of a completed design cycle, 
parts of which were carried out manually, applied to a specific course in Air Force 
technical training. 

This work has been conducted under a Rand project entitled Analysis of Systems 
for Air Force Education and Training. Emphasis has been on the use of technology 
in designing instruction for formal technical training or jfor higher education, as at 
the Air Force Academy. The results will support the activities of the Director of 
Personnel Plans, Headquarters USAF; DOS/Technical Training and the Training 
Development Directorate, Headquarters Air Training Command; and the Air Force 
Human Resources Laboratory, especially the Technical Training and Professional 
Education divisions. It will be of particular interest to those working on the Ad- 
vanced Instructional System. 

This report is part of a continuing Rand effort to apply systematic methods of 
analysis and synthesis to issues and problems in education and training. Related 
studies have concerned Air Force pilot training and management of the pilot force, 
evaluation of programs of compensatory education, design of information systems 
for local school districts, and other diverse concerns. A special bibliography of Rand 
work in education is available on request. 
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R-L020-PR THE MODIA QUESTIONNAIRE FOR CURRICULUM ANALYSIS, Rudy Bretz, 
November 1972, Unclassified 



The Professional Education Division of the Air Force Human Resources 
Laboratory, mentioned in the Preface, was disbanded xn July 1972 
and no longer exists as an organizational unit. 



SUMMARY 



This is one of four reports describing Rand work for the Air Force on the 
development of methods for designing programs of instruction, particularly for 
technical training and higher education. The first report in the series gives an 
overview of Rand's comprehensive, semiautomated method, called MODIA (A 
Method of Designing Instructional Alternatives).^ The second and third in the series 
describe some of the major analytical tools in the MODIA process; and the fourth 
gives the results of a sample design cycle, completed for a specific course in Air Force 
technical tra!ining. 

This is the third report in the series. It presents, and discusses the use of, a 
questionnaire that helps the designer of a curriculum analyze a course of study to 
gauge its requirements for communication media, personnel, and time. Use of the 
Curriculum Analysis Questionnaire (CAQ) figures as a middle step in MODIA, which 
involves the following procedures: 

Step 1. Analyze the learner population. 

Step 2. State the policy (objectives) of the educational institution. 
Step 3a. Analyze the curriculum Q^y means of the CAQ). 
Step 3b. Specify the instructional strategy (computer-assisted). 
Step 4. Specify the design criteria. 

Step 5. Describe the rate of student entry and the institution's resources and 
constraints. 

Step 6. Produce an instructional design alternative (computer-automated). 

Step 7. Produce a cost and resource analysis (computer-automated). 

Step 8. As necessary, depending on the acceptability of the outputs, repeat 

any of steps 1-7 with different inputs until the most desirable system 

emerges. 

Curriculum analysis by means of the CAQ works by causing the user of the 
questionnaire (the designer, who has already established his curriculum and knows 

* For the rationale and general description of the procedures and tools involved in MODIA, see Polly 
Carpenter, An Overview of MODIA: A Method of Designing Instructional Alternatives for Air Force 
Training, The Band Corporation, R-1018-PR, November 1972. 



what is to be taught) to subdivide each course inco basic ''learning events,*' units of 
homogeneous instruction, and to proceed through the CAQ for each one. Since the 
CAQ is structured as a branching questionnaire, the user encounters only those 
questions that are relevant to the particular type of instruction that the learning 
event represents. The user is led to specify characteristics such as the learning 
event's requirements for personnel and facilities, and its learning objectives and 
appropriate means of diagnosing their attainment (e.g., multiple-choice questions, 
or a student performance or product). 

Each learning event is classified in one of twelve instructional categories falling 
in three general types of instruction. Type I instruction refers solely to the presenta- 
tion of information and requires no drill or practice; Type II instruction requires 
drill Ox' practice (either cognitive or psychomotor) but can be done in a classroom or 
carrel without special facilities; and Type III instruction requires special equipment 
or facilities, such as a laboratory or workshop. 

Besides analyzing the learning events in his course in increasing detail as he 
proceeds through the CAQ, the designer discovers which learning events could 
benefit from communication media and what classes of media are appropriate to 
them. (Whether or not he chooses to use media at all for any learning event is a 
decision that conies later in the process of instructional design.) 

To illustrate how the CAQ guides the designer in the choice of communication 
media, questions following the logic of a simple decision tree ask the designer 
whether the subject to be taught in the learning event is abstract or concrete, and 
if concrete, whether it requires visual or audial (auditory) presentation, or both. The 
CAQ guides the user to the siniplest appropriate communication media; For exam- 
ple, if a subject does not require audial presentation, the expensive audio-motion- 
visuai media (e.g. , television) are eliminated; if it does not need to be seen in motion, 
the motion-visual medium, silent film, is also eliminated; the final appropriate 
media class is thus the still-visual (e.g., the printed page). 

It should be noted that the CAQ indicates communication options by class of 
media, according to a taxonomy developed by the author (see inside front cover). The 
final choice of media to be used in a course should specify a particular medium (e.g., 
film, video tape, live television), submedium (e.g., 16-mm, 8-mm, 2-in. video tape, 1-in. 
tape), mode (group or individual presentation), and response or feedback system 
required. Details of the most practicable media system will emerge from step 6 of 
the MODIA process, after the institution's instructional criteria and its resources 
and constraints, have been taken into account. i 

It is intended that the CAQ be tested in real-world course design. The final 
version that emerges, though an integral part of the semiautomated MODIA process 
will be able to be used, independently, in a purely manual process of instructional 
design. In either case the CAQ is used manually. The information elicited by the 
CAQ is needed for designing ari instructional system, regardless of the method being 
followed. With a few additional questions, the CAQ can provide even the planner of 
a traditional instructional system with a convenient means of analyzing his instruc- 
tional needs. 
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I. INTRODUCTION 



Since 1968, Rand has been working on the design of instructiorial^rograms'td 
help the Air Force, especially the Air Training Command (ATC), determine sys- 
tematically the mix of instructors, facilities, materials, and students that should go 
into its training and other instructional programs. As a result of that work, Rand 
has developed MODIA, a comprehensive methodology for designing an instructional 
program. MODIA (A Method of Designing Instructional Alternatives) consists of a 
sequence of procedures andsemiautomated**tools,"some of which have already been 
designed and are described in other reports in this series.^ 

Figure 1 illustrates the steps in the MODIA process. As it shows, the process is 
intended to produce precise guidelines about how long a course should be, the 
number of students who should constitute a class, and what facilities, equipment, 
personnel, and dollars are needed, and when. By means of MODIA, planners should 
be able to design an instructional program efficiently and with confidence both that 
the guidelines are acceptable to the teaching institution and that the methods of 
teaching the course are appropriate tO the learners. The steps are briefly described 
below. 

Step 1: Analyze the learner population in terms that will affect the way the 
course is taught (e.g., will some students be more familiar than others with the 
subject matter?). 

Step 2: State the institution's policyhy means of a short questionnaire,^ which 
elicits information on the institution's goals, the extent to which it can adjust to 
variations in the student population, and how it relates to the institutions that 
supply its students and to those that use its graduates. 

Step 3a: Analyze the curriculumhy means of the CAQ, a branching question- 
naire that helps the user describe his course of study in detail. It elicits a description 
of eaeh learning event^ in the course of study in terms that help determine requu-e- 

* See the list in the Preface. 
Polly Carpenter and Barbara Homer, The MODIA Decision Process for Deueloping Strategies of Air 
Force Instruction, The Rand Corporation, R-1019-PR, November 1972. 

" See Glossary. Throughout, terms listed in the Glossary appear in boldface type the first time they 
are used. 
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merits for personnel, communication media, and other appropriate resources. The 
CAQ is structured so that for each learning event the designer encounters only 
questions that are relevant to the event's particular type of instruction. After the 
designer has gone through the CAQ for each learning event, he is prepared for step 
3b, where specifications of instructional method are explicitly entered into the 
design process. 

Step 3b: Specify strategies of instruction by means of a logic tree, with each 
decision point a logical consequence of the preceding decisions.® This step employs 
DISTAF, an interactive computer program written for use on JOSS^ that is being 
translated into Interactive Fortran. Its user's manual presents the pros and cons of 
the decisions to be made at each point, along with some of the logical consequences 
of each choice. DISTAF permits the designer to formulate dozens of different teach- 
ing strategies for a single course. As defined here, '^instructional strategy" has two 
basic dimensions. For each type of instruction (identified in the CAQ), the designer 
specifies what the teaching agent will be and how students will interact with the 
agent. 

Step 4: Specify and rank design criteria, represei^ting the institution's instruc- 

" Carpenter and Horner, op. cit. 

® Rand's on-line, interactive computer system. 
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tional desiderata, such as least cost, shortest course length, graduation of the most 
students per unit of time, and maximum use of communication media. 

Step 5: Describr local resources^ information on the rate of student entry and 
the school's resources and constraints, to be gathered by means of a logically struc- 
tured set of questions. Data include number of available classrooms, laboratories, 
and other facilities; the avail -ability of communication equipment; and the geo- 
graphic distribution of students. The resulting description is used in step 6 to deter- 
mine class size, to select specific communication media, and to serve other purposes; 

Step 6: Produce an instructional system. The final design will incorporate as 
direct inputs the curriculum-analysis data, the instructional strategies, the design 
criteria, and the description of local resources. First, each learning event is linked 
to the instructional strategy chosen for its particular category of instruction. Sec- 
ond, student flow through the course is simulated by a flow and scheduling model 
to derive graduation rates and resource requirements. Third, specific personnel, 
communication media, and facilities are selected. 

Step 7; Analyze the costs and resources of the system. 

Step 8: As necessary, depending on the acceptability of the outputs, repeat any 
of steps 7-7with different inputs until the most desirable system emerges. 

This report describes a key element in the MODIA process (though it can be 
used separately) — the Curriculurn Analysis Questionnaire (CAQ). Used as part of 
MODIA, it comes into play in step 3a (see Fig. 1). Whether incorporated in MODIA 
or separate, however, the CAQ is intended for use by an instructional designer or 
planner when he knows what the substance of the proposed course will be and who 
will take the course, and must decide how to teach it. 

The "how" that the CAQ particularly addresses is the use of communication 
media in the course. Communication media have become increasingly important in 
instruction, but many course designers are not familiar with their use. The CAQ 
provides guidance on whether communication media are appropriate at all for a 
given course, and if so, what kind. It should be noted that that guidance is limited 
to those communication media classes or specific media (according to the author's 
taxonomy reproduced inside the front cover) that are appropriate to the instruc- 
tional need. Decisions about what specific media or submedia are practicable to use 
in a course are properly made later in the instructional design process, after inputs 
on resources and constraints have been entered into the process. 

Section II discusses the rationale and features of the CAQ, and Sec. Ill re- 
produces and explains the text of the questionnaire. Section IV gives a hypothetical 
example of use of the CAQ to analyze a course in beginning photography. Then, in 
Sec. V, suggestions are given for field testing the CAQ with courses actually being 
designed. 
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II. WHAT THE QUESTIONNAIRE DOES 



The Curriculum Analysis Questionnaire has two features new to instructional 
design. The first is a new taxonomy of instruction. The second is a logic process for 
selecting the simplest appropriate class of coreim mication media according to the 
subject matter and the category of instruction in each learning event. 

As with any tool, the success of the CAQ depends on the skill and judgment of 
the user. A brief glance at some of the questions reveals that the judgment of the 
user is especially important in answering the questions concerned with choosing 
communication media. Thus, the CAQ does not eliminate the need for a highly 
trained and experienced instructional designer who is thoroughly familiar with the 
resources and constraints of the teaching institution. The designer's background 
should include knowledge of conventional teaching methods and at L?ast one com- 
munication medium. 



ANALYZING INSTRUCTION 

At the point in the development of an instructional system when the CAQ is 
used, the designer knows exactly what is to be taught but not how it should be 
taught. Instructional technology has provided many alternative methods by which 
the student ca.i receive instruction. Presention of theory can be made by an instruc- 
tor in a classroom, by a projected film or video tape, or by an individual communica- 
tion device, such as a film viewer or an audio cassette player. Likewise, the perform- 
ance model (demonstration of a skill to be learned) can be presented to the student 
in many ways. Student performance can take place on a fully operating machine, 
on a functioning mock-up containing only the parts necessary to the immediate 
objective, or on a simulator. 

Evaluation, like presentation, can be done many ways. Formal evaluation can 
be made directly by an instructor or indirectly by recording student performance for 
later assessment. If the evaluation need not be a formal measurement of achieve- 
ment (e.g., an examination), it can. be done by the student himself, by comparing a 
record of his performance with a model of the required standard. 
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The CAQ helps the instructional designer decide among these many options. 
First, he must divide his course into elemental units of instruction called learning 
events. For example, in a course designed to teach how to operate a machine, the 
first learning event might include the theory behind the equipment and the interre- 
lation^ship of its parts; the names of the controls and how they are related to the 
functions they perform. That learning event could usually be accomplished in one 
classroom lecture or lecture-discussion. The next learning event in such a sequence 
»night be a performance model, where the instructor v/ould demonstrate the equip- 
nt in operation. The student would observe the procedure he would later follow, 
techniques it involves, and the criteria that define the performance expected of 
him. Normally, a third learning event would take place, of considerably longer 
duration, in which the student would take over the controls and practice ufjing the 
equipment himself That event might require the presence of an instructor to assess 
the student's performance or to prevent injury to the student or damage to the 
equipment. 

The ultimate intent of identifying learning events is to estimate the kinds and 
amounts of resources that are needed to accomplish each part of a course of instruc- 
tion. The term "learning event" refers only to basic instruction, not, for example, 
to question-and-answer or review sessions. The resource requirements for these 
noninstructional activities can be calculated by reference to the learning events on 
which they are based. 

The instructional designer analyzes a course of study by answering the appro- 
priate questions on the CAQ for each learning event, one event at a time. His 
responses, entered on a form and later transferred by a card-punch operator onto 
electronic data-processing cards, constitute the CAQ input to the MODIA system. 
Even if MODIA is not involved, standard data-processing procedures can be used to 
determine, for example, the total amount (i.e., for all learning events and instruc- 
tional minutes) of each type of resource and the amount of program material re- 
quired for each medium or class of communication media. 

Because the purpose of curriculum analysis by means of the CAQ is the practi- 
cal determination of resource requirements, rather than the theoretical classifica- 
tion and description of learning, existing systems for classifying instruction could 
not be used. That includes the conventional division of instruction into knowledge, 
skills, and attitudes, even in the form set forth in Bloom's Taxonomy of Educational 
Objectives, where the three divisions are called the cognitive, psychomotor, and 
affective domains.^ 

First, we had to distinguish between instruction that requires only the presen- 
tation of information (with or without student response) and training in skills, which 
requires drill or practice. The simplest classification systems fail to recognize that 
not all skills are motor or psychomotor; many are purely cognitive (e.g., computa- 
tional skills, language vocabulary recall, skills of perception and recognition). Since 
our concert was with the need for resources, whether drill or practice is cognitive 

Benjamin S. Bloom et al., Tuxonomy of Educational Objectives: The Classification of Educational 
Goals, New York, David McKay, 1956. 
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or psychomotor seemed less relevant than whether it can be done in a classroom or 
carrel, or whether it requires special facilities (e.g., parade ground, hangar, labora*- 
tory) or special equipment (e.g., musical instrument, a piece of machinery). 

For the purposes of the CAQ, we divided instruction into three general types, 
which are in turn divided into categories (see Table 1). Type I covers instruction not 
requiring student performance (drill or practice). It refers to the simple presentation 
of facts and ideas for students to learn or be moved to (later) action by. This type 
of instruction may have affective goals. Students are expected to absorb Type I 
instruction, not to perform, beyond perhaps answering questions. Type II includes 
instruction in skills that the students are expected to master. It requires drill or 
pracuce in a classroom or carrel without special equipment or facilities. Type III 
instruction requires the students to learn skills that demand special equipment or 
facilities or both. The 12 subtypes, or categories, are important to the utility of the 
CAQ, and a key question asks the user to assign a number denoting category of 
instruction to each learning event. 

Category 1 instructiori represents the typical instance of knowledge or theory 
presentation, usually in the form of a classroom lecture. Category 2 instruction 



Table 1 



CLASSIFICATION OF INSTRUCTION IN THE CAQ 



Type 


Category 


I 

9 no student performance 


1. Presentation not requiring special 

equipment or^ facilities 

2, Presentation requiring special equip- 

ment or facilities 


II 

9 student performance 

• requires no special 
facilities or equip- 
ment — can be done in 
classroom or carrel 


3. Pure performance, interactive skills 

4. Pure performance, individual skills 

5. Performance model, interactive skills 

6. Performance model, individual skills 

7. Follou-me (mixed demonstration and 

performance) , individual skills 


III 

• student performance 

• requires special 
facilities or 
equipment 


8. Pure performance, interactive skills 

9. Pure performance, individual skills 

10. Performance model, interactive skills 

11. Performance model, individual skills 

12. Follow-me (mixed demonstration and 

performance) , individual skills 
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requires special equipment or facilities for presentation, but does not include stu- 
dent performance. It may consist of the demonstration of phenomena or equipment. 
Category 1 and 2 instruction is entirely amenable to the use of communication 
media and associated response systems. Instruction of the following categories may 
use these media less extensively. 

Category 3 instruction involves student performance of interactive skills, either 
simple or complex, that can be exercised in a classroom. Such skills include choral 
singing, dramatics and role-playing, and various instructional- games. Category 4 
refers to drill and practice in individual rather than interactive skills requiring no 
more special equipment than pencil and paper or small objects. Categories 3 and 4 
are characterized as "pure performance" because no demonstration is involved. 
Categories 5 and 6 also refer to interactive and individual skills, respectively. How- 
ever, they involve a perforniance model— demonstration only. Though they are 
intended to generate later student performance, performance is not an immediate 
part of these purely presentational categories. Category 7 includes instruction in 
v^hich a performance is alternately demonstrated by an instructor and then per- 
formed by a learner, step by step. Categories 8 through 12 are similar to the preced- 
ing five categories but fall in Type III: instruction involving drill and practice but 
located elsewhere than a classroom or carrel. 

Most of these 12 categories can be subdivided into simple and complex instruc- 
tion and, further, into instruction in which student progress can be checked by 
selected responses (e.g., multiple-choice questions) and that requiring con- 
structed responses (e.g. f written, spoken, manipulative). 



CHOOSING THE APPROPRIATE COMMUNICATION 




After the designer has specified the types of instruction in the learning events 
of the course he is analyzing, the CAQ helps him choose the simplest class of 
communication media appropriate for each event.® Decision tables in the CAQ ask 
a number of questions that are critical in the selection of communication media and 
give detailed criteria for their answers; the pattern of answers designates the appro- 
priate media class. To illustrate, we find the following questions: "Are visual means 
required?" ''Is sound required?" "Is motion required?" If the answer to all of them 
is "yes," class I, audio-motion-visual media, is indicated. The class indicated in the 
example just cited includes specific media such as sound film (in sizes ranging from 
Super-8 to 16-mm), presented on a large screen or an individual viewer; live televi- 
sion or video tape in a number of diflTerent formats; and the new video cassettes. 

® For a more detailed discussion of the classification of communication media, see Rudy Bretz, A 
Taxonomy of Communication Media, Englev/ood Clifts, N. J., Educational Technology Publications, 1971 
(also available in soft cover to Air Force users as Rand R-697-NLM/PR). A basic discussion of the 
instructional uses of communication media is found in Rudy Bretz, The Selection of Appropriate Com- 
munication Media for Instruction: A Guide for Designers of Air Force Technical Training Programs^ The 
Rand Corporation, R-^l-PR, February 1971. 
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To narrow the choice of communication medium, the decision tables are fol- 
lowed by other questions that ask the user to determine the level of visual and audio 
fidelity required if the instruction were to be done by communication media rather 
than face-to-face presentation. For example, the user is asked to state a learning 
event's needs for visual fidelity on three dimensions: color, range of gray tones, and 
degree of definition required. If he states that he must have full and realistic color, 
this requirement not only eliminates all black and white media but, in the case of 
class I media, most color television and video tape systems as well. Though the latter 
are capable of full color, it is not necessarily or consister Iv ron!?*^tk s^VSi . If he 
indicates that black and white pictures would suffice, colOi by):3i^!fns are Miminated. 
(Black and white pictures can be carried over a color system, but the CAQ is designed 
to indicate the s/m/jfes^ appropriate medium.) However, if many learning events in 
the course require color, and only one calls for black and white, the color system 
alone would be indicated, since it would make little sense to use two separate 
systems. 

Thus, the CAQ prepares the way for final instructional system design by sug- 
gesting one or more appropriate communication media for use in a course. The final 
selection of specific media is made later in the instructional design process, on the 
basis of available resources and existing constraints. If the MODIA methodology is 
being followed for the instructional design process, that decision occurs in step 6, at 
which point the choice of specific communication media for particular courses is 
almost automatic. 
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III. THE CURRICULUM ANALYSIS QUESTIONNAIRE 



PRELMN ARIES 

T^efoEe the is used, the following tasks should be completed. 

1. The curriculum of the course should be planned with the objectives spe- 
cified, preferably in behavioral terms. 
2r Tlie objectives should be in logical sequence. 

3, The instruction should be iSvided- into topics and the topics subdivided 

into learning ejvents. 
4- Ah approximate time: shouM be allotted to each learning event. 

Estimates HEay beibased on prasdous experience with the course of study. Data are 
required oi% for ttjnie learning:«vents. Time allotments should not include: 

• Aisswering students'i^estions about previous instruction or discussing 
wMi students subject matt^ previously covered.® 

« SoG0ringfi3©skwori£ oir homework in cIelss. 

• Reviewik^ previous uaKtructioii. 

• Formal CTaminations of student achievement. 

• Housekaeeping (announcements, cleanup, etc.). 

• Coffee breaks, transportation, etc. 

Such activities aremot considered partiof the curriculum and do not require analy- 
sis. If the CAQ is issed as part of M0DIA, the resource requirements for those 
activities are autaamatically calculated on the basis of (1) the established instruc- 
tional strategy and <|2) the questionnaire data concerning the true learning events 
from which the fersgoing noninstrucitional activities derive. 

If the CAQiBJia^ independently of MGDIA, time and resources for noninstruc- 
tional activities aEedeterminedLsimilarly, with the designer taking his strategy and 
the original learning events into account. 

^ irthe aim of a diafetassion session is to deveiop students* skills in conversation or group relationships, 
however, the discuaeip^^ session should be conadered a learning event. 



The questionnaire is divided into three parts. Part I is intended to help the 
instructional designer specify his course in increasing method-related detail. Part 
11 IS intended to help him decide whether communication media are appropriate for 
any parts of it, and : so, which parts and what class of media. Part III is intended 
to help him define more precisely the specific communication media that are appro- 
pnate. 

The questionnaire appears as three columns. The first column cues the user 
which questions to answer for each learning event. The second column contains the 
question proper and the various possible responses. The third column lists the code 
numbers for the answers, one of which should be entered on the response form for 
each question. 



PART I 



Ansuer foT 

k All events 
j All events 
1. All events 



2. All events 



3. All events 



A. All events 



Question 
Assign a topic number. 
Assign a learning event number. 

Estimate the number of minutes that would 
be required for the average student in 
your student population to achieve the 
objectives of this event, given conven- 
tional instructional methods* and no time 
spent waiting for facilities or equipment 
to become available? 

Estimate the number of minutes that would 
normally be* devoted to housekeeping* ac- 
tivities for this event. 

Does the accomplishment of this task re- 
quire the integration of several simpler 
concepts or skills? 

Yes: 
No: 

Considering the capabilities of the aver- 
age student in your learner population, 
how difficult is this event to master? 

Relatively hard: 
Average or relatively easy: 



Code 



k = 
j = 



x(l) 
x(2) 



x(3) = 1 
Jt(3) = 0 



x(4) = 1 
x(4) - 0 



min 



min 



auestlonn.?^!"^''\^°^ definition. An, asterisk will be used throughout the 
s%":cJine1nlng: S^JlslonS^i:" ^"'^^ ^^^"^ ^° °' ^'^'^ 
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Ansuer for 
5. All events 



6. All events 



Question 

Is this learning event intended to be 
encountered by all students? 

If so. 
If not: 

It is intended for students who lack 
necessary prerequisites. 

It is intended to be skipped by 
students who have shown mastery of 
the subject matter. 

It is intended to be skipped by 
students who are not bright enough 
to grasp the subject matter in the 
allotted t ime . 

It is intended to be skipped by 
extremely bright students. 

It is intended as enrichment for 
students who finish other work easily 
and have additional time. 

It is intended as remedial instruction 
for studentr who have difficulty master- 
ing the assigned subject matter. 

Indicate the le,:)rning event's category of 
instruction. [See Table 1.] 

Type I (presentation only): 

No special equipment or facilities 
required 

Special equipment or facilities 
required 

Type II (in classroom or carrel) : 

Performance of 

Interactive skills 
Individual skills 

Performance model of 
Interactive skills 
Individual skills 

Follow-me instruction (individual 
skills) 

Type III (requires special facilities) : 

Performance of 

Interactive skills 
Individual skills 

Performance model of 
Interactive skills 
Individual skills 

Follow-me instruction (individual 
skills) 



Code 

x(5) = 0 
x(5) = 1 
x(5) = 2 

x(5) = 3 
x(5) - 4 

x(5) = 5 
x(5) = 6 



x(6) = 1 
xC6) = 2 



x(6) = 3 

x(6) - 4 

x(6) - 5 

x(6) = 6 

x(6) = 7 



x(6) = 8 
x(6) - 9 

x(6) - 10 
x(6) - 11 

.x(6) = 12 
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EKLC 



Ansuer for 
events for 
uhioh x(6) 

7. 3,4,7,8,9, 
12 



Question Code 

Is the achievement of a minimum rate of 
performance (speed) an objective of this 
learning event? 

Yes: x(7) 
No: x(7) « 



8. 3,5,8,10 



9. 2,5,6,10,11 



10. 3,4,7,8,9, 
12 



11. 8,9,12 



12. 2,8,9,10, 
11,12 



13. 3,4,8,9 



14. 3,4,8,9 



In this event requiring interactive per- ^ 
formance, how many are needed in the 
group? 

Hov7 many persons besides the instructor 
are required to produce the presentation, 
whether face-to-face or recorded? Enter 
number by level . 

a. Instructor level 

b. Student level 

Must an instructor be present to evalu- 
ate the student *s performance and provide 
him with knowledge of results? 

Yes: 
No: 

Must the student be supervised to pre- 
vent injury to himself, the equipment, 
or the facilities? If yes, by whom? 
(If x(10) = 1, enter 1.) 

Not necessary 
Instructor 
Instructor aide 
Student monitor 

Wliat special area is required (for face^ 
to-face presentation, student performance, 
or production of media materials)? 

None 

A distant location 

Outside but near school 

Large laboratory, workshop, etc. . 

Small. laboratory 

Can- student progress in mastering the 
skill in this event be adequately 
diagnosed by selected responses* (e.g., 
multiple-choice questions) ? 

Yes: 
No: 

Answer only if x(13) =0. What con- 
structed response* is required? 

A simple verbal (written or spoken) 
expression 



x(8) 



x(9a) = 
x(9b) = 



x(10) = 1 
x(10) = 0 



x(ll) = 0 

x(ll) = 1 

x(ll) = 2 

x(ll) = 3 



x(12) 
x(12) 
x(12) 
x(12) 
x(12) 



x(13) = 1 
x(13) = 0 



x(l4) = 0 
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Ansuev for 
events for 
which x( 6) = 



QuvQst.i on 



Code 



A student performance (including paper- 

and-pencil problem-solving) x(lA) = 1 

A student product (e,g,, object, essay, 

photograph, film, tape, etc.) x{lA) = 2 

15. 3, A, 8, 9 Answer only if x(13) = 0. What is the 

simplest appropriate means for recording 
the constructed responses required in 
this learning event? 

Paper and pencil 3c(15) 1 

Keyboard and co'iiputer memory x(15) = 2 

Audio tape x(15) = 3 

Still photography x(15) = A 

Silent film x(15) = 5 

Video tape (sound and picture) x(15) = 6 

Other materials x(15) = 7 

16. 2,8^9,10,11, Is equipment other than cameras.^ tape 

12 recorders, etc,, required to make the 

presentation or for use in student 
performance? 



17. 2,8,9,10,11, Answer only if x(16) = 1. Is the 
12 special equipment already on hand? 



Yes: x(16) - 1 
No: x(16) = 0 



Yes: x{17) = 1 

No: x{17) = 0 



18, 2,5,6,7,10, If the presentation were given face- 

11,12 to-f ace .with the usual instructional 

aids, how many people could see it at 

one time? x(18) 

19. 1,2, A, 5, 6, Estimate the length of time needed to 

9,10,11 make the presentation, give the demon- 
stration, or specify problems and direc- 
tioiis'. This is meant to exclude time 
spent in class discussion, answering 
students' questions, student performance, 
and housekeeping. If presentation must 
be repeated, estimate time for first pre- 
sentation only, x(19) = min. 
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Ansuer for 
events for 
uhich x(6) 

20. 3, A, 7, 8, 9, 
12 



Ansuer^jfor 

21. All events 

22. All events 



23. All events 



24. All events 



25, All events 



26 . All events 



27 . All events 



28. All events 



Question 

Are special materials required by 
students for use in performance? 



Yes 1 
No: 



Is this a scheduled or an unscheduled 
learning event: 

Scheduled : 
Unscheduled : 

Will there be a subsequent formal 
evaluation of the student's mastery of 
the subject inatter in this learning 
event? 

Yes : 
No: 

Estimate the number of months between 
revisions of the curriculum content .of 
this learning event. 

Does this learning event contain classi- 
fied information? 

Yes : 
No : 

Is this event at the end of a major 
section of the course? 

Yes : 
No: 

Answer if x{25) ^ 1. Estimate the per- 
centage of students who will drop out 
or be eliminated from the course before 
the next learning event. 

Answer if x(25) = 1. Estimate. the per- 
centage of students who will be washed 
back to repeat the last major section. 

Estimate the percentage of students who 
, would normally need some remedial in- 
struction after this learning event. 



Code 



x(20) = 1 
x(20) = 0 



x(21) = 1 
x{21) = 0 



x(22) = 1 
x(22) = 0 



x(23) 



x(24) = 1 
x(24) = 0 



x(25) = 1 
x{25) = 0 



x(26) 
x(27) 
x(28) 



PART 11 

Part 11 of the CAQ, comprising the complex question 29, is intended to help the 
instructional designer determine which learning events in his course can benefit 
from communication media, and what classes of media are appropriate for them. It 
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docs so by guiding him through two decision tables, A and B, which distinguish 
between subject matter in the ''perceptual" and "conceptual" domains. These terms 
correspond approximately to "concrete" and ''abstract," except that not everything 
that is perceived is generally considered concrete (e.g., optical phenomena, actions, 
and events). 

The perceptual domain includes subjects that can be perceived and directly 
experienced through the senses (e.g., persons, places, and things). Realistic represen- 
tation is often required for presentation of such subject matter, to give the learning 
greatest transfer value when the student experiences the subject in the real world. 
Thus, audiaP° or visual means or both are generally required of comipunication 
media to represent subjects in the perceptual domain. (If perception is required by 
senses other than sight and hearing, instructional aids may be used, but existing 
communication media will not be adequate since they involve only the two senses.) 

The conceptual domain includes concepts, thoughts, and mathematical rela- 
tions. Although presentation of information in that domain must generally be done 
with words or other symbols, it is often desirable to support the effect of words with 
print or picture. Evidence has shown, however, that for representing concepts, 
abstract depictions— stick figures, line drawings — are often more effective than real- 
istic representations such as photographs. 

By proceeding through Decision Tables A and B in question 29, the user is led 
to select the simplest appropriate class of communication media for the presentation 
of each learning event in his course. As noted above, the fact that a media class is 
designated does not necessarily mean that it will be used. That decision occurs later 
in the MODI A process. 

At that later point, in determining the most practicable communication 
medium needed, the MODIA design process will raise the level of quality or capabil- 
ity above what the user specifies whenever possible. It will not lower it below the 
minimum the user indicates to be acceptable. Therefore, when making the decisions 
required in the tables, the user should specify only the capability or quality required 
(unless the questions ask for "desirable" capabilities). He should not ask for motion 
pictures, for example, if still pictures will achieve the objective, or for color images 
if black and v/hite will do. Instructional designers often call for motion, color, high 
definition, and the like — even though they are not required for the specified objec- 
tives— with the intent of thereby changing students' attitudes. If such changes are 
desired, the statement of objectives of the learning event should be rewritten to 
include these affective goals. q 

Learning events that are pure performance (x(6) = 3, 4, 8, or 9) do not involve 
presentation and would seem not to need communication media. However, media 
might be useful for the communication of the necessary directions and stimuli for 
drill or practice. Against the possibility, then, that communication media will later 
be specified, the user should assign a class of communication media to learning 

The word "audial" is coined as the counterpart of "visual." The nearest existing words are ''audi- 
tory" and "audio," but neither conveys the sense intended. "Auditory" implies a reference to the physi- 
oIog>- of sound perception. "Audio" refers to physical sound or its reproduction, especially when tran- 
slated into electrical signals. Neither is quite right for denoting a characteristic of program software. 
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Ansuer for 
29. All events 



DECISION TABLE A 



Conditions: Does this learning event require: 




Rules (Read doun.) 




1. Visual means besides print?^ 


Y 


Y 


Y 


Y 


N 


N 


N 


2: Sound?^ 


Y 


Y 


N 


N 


Y 


N 


N 


3. Motion?^ 


Y 


N 


Y 


N 


„d 






4. None of the above. Instead, perception 
through senses other than sight and 
hearing?^ 












N 


Y 


Actions: 


5. Simplest appropriate media class [x(29) ] 


I 


II 


IV 


V 


VII 




of 


6. Go to Decision Table B 












X 





Visual recognition or discrimination is an objective of the 
learning event . 

A process or procedure is to be presented or demonstrated. 
Two-dimensional relationships are important (physical place^ 
ment , relative location). 
Three-dimensional structure is important. 
Visual art Is being studied. 

Specific sounds must be presented. 
Music or oral literature is being studied. 
Presentation merely in textual form would make the subject 
difficult for many students to understand. 
Simultaneous aural narration would be preferable to inter- 
rupting visual continuity with caption frames. 

Mastery of the instruction will eventually require the recog- 
nit ion or performance of a procedure that Is initially un- 
familiar to the learner, 
o The subject's manner of movement is Important to, the recogni- 

tion or description of it. 
o Slow or fast motion is required to portray changes that ordi* 
narily take place too rapidly or too slowly to be otherwise 
comprehensible . 

^Dash means that either "Y" or "N" would give the same rC3ult, 

^Answer "Y»" for example, if the subject matter is the learning of 
certain darkroom techniques, which require only the sense of touch. 

^None appropriate. 



Answer 
o 

o 
o 

o 
o 

^Answer 
o 
o 
o 



"Answer 
o 
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AnsDur for 

29., Learning events indicated by Decision Table A 
cont. as being in the conceptual domain. 

DECISION TABLE B 



Conditions: Would the following he 
desirable in your learning c}^ent? 


Rules^ (Read doun.) 


1. Presentation by audial means 
rather than print?** 


Y 


Y 




Y 


Y 


Y 


Y 


N 


N 


N 


N 


N 


2. Print or visuals to accompany 
audial present ation?b 




Y 




Y 


Y 


Y 


N 


Y 


Y 


Y 


Y 


N 


3. Pictures, In addition to alpha- 
numerics or line graphics?^ 




_c 




N 


N 


Y 








N 


Y 




^. Semimot ion?^ 




N 




N 


Y 


Y 






N 


Y 


Y 




5. Full motion?^ 


Y 


N 




N 


N 


N 




Y 




N 


N 




Actions : 


6. Simplest appropriate media class 
[x(29) =] 


I 


II 






III 




VII 


IV 


V 


VI 




VXII 


7. Simplest appropriate multimedia 
categoryS [x(29) =] 






VII 


-VIII 




II-lII 










V-VI 





NOTE: As mentioned in tho text, subject matter in the conceptual domain requires 
only spoken or printed words and symbols for communication* But visual communication 
means may be desirable . Determining what they are is the purpose of this decision 
table. 

^Answer "Y" if: 

o Presentation in print only would make the subject difficult for many 
students to grasp. 

o The personal element (more readily conveyed by the spoken word than by 

print) is important in this learning event, 
o An objective of this learning event is to change feelings or attitudes, 

and the effective use of audial means would help achieve it. 

^Answer "Y" if: 

o This learning event Involves concepts and relationships that would be 

clarified by graphic presentation (charts, graphs, diagrams, etc.). 
o The logical relationships or order of items is important enough to 

warrant having a diagram or outline to refer to throughout the presentation, 
o Reading or writing is being taught, or a difficult technology is being 

introduced, and spoken words should be related to their counterparts In 

print. 

^Answer "Y** only if: 

o A value concept is to be Illustrated by showing its indirect results 

(e.g., an emotion by facial expression or gait), 
o A concept is to be Indicated by a concrete symbol (e.g., handshake = 

friendship; flag « country, patriotism). 

If the answer here is "N," a choice of media classes is offered: class II or 
multimedia category VII-VIII. 

^Answer "Y" if: 

o The information would be more comprehensible by writing words or other 
symbols in sequence, or by drawing diagrams, e.g.» on a chalkboard. 
The motion of pointing would be useful, e.g., for tracing a route on a map, 

o The concrete effects of hidden or abstract processes are to be shown, 

and motion can make the processes perceptible. — 
o 'Animation of diagrams or other symbolic representations is needed to 

demonstrate the effects of change , either in the symbols themselves (e^g., 
the movement of letters to teach reading) or in the idea they represent 
(e.g., animating an organizational chart to show a change in government 
structure) . 

^For multimedia classes, see Table 2. 



f 



Answer 
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events in those categories. In most cases, it will be class V (still-visual), which 
includes printed matter. 

To ansv/er question 29, begin vi^ith Decision Table A. Reading the questions in 
the Conditions column, answer each **Y'' or "N" ("Yes" or "No"). The footnotes to 
the questions give criteria for making the decisions. Then, look over to the Rules 
column and find the pattern of Y-N answers that corresponds to that for your 
learning event. In most cases, Action 5 will tell you the number of comraumcation 
media class that is appropriate for your learning event. That number is to be entered 
on your response fon?< as the answer for question 29. A pattern of "N'V answers to 
all four questions me zns that the subject matter':?of the learning event is in the 
conceptual domain, ai^ you are directed to DecisioiiTable B, which is read similarly 
to Decision Table A. Besides specifying class of media, Decision Table B also specifies 
class of multimedia, where appropriate (see Table 2). 

Table 2 

CLASSES OF COMMUNICATION MEDIA AND MULTIMEDIA CATEGORIES 



Media Classes 

I Audio-mot ion-visual 

(e.g. , sound film, TV) 

II Audio-still-visual 

(e,g*, sound slide set) 

III Audiorsemimotion 

(e.g. , telewriting) 

IV Motion-visual 

(e.g. , silent film) 

V Still-visual 

(e.g. , printed page) 

VI Semimotion 

(e.g. , telcautograph) 

VII Audio 

(e.g. , audio tape) 

VIII Print 

(e.g., teletype) 



I-II 



II-III 



IV-V 



V-VI 



Multimedia Categories 



Audio-motton-stlll-visual (e.g. , 
Norelco's P. I. P., a sound-motion- 
fiimstrip system)^ 

Audio-semimotion-still-visual (e.g. , 
telewritevision) 



Mot ion-st ill-visual (e.g., Eastman 
Kodak's MFS-8, a motion-film- 
strip system)^ 

Semimotion-stlll-visual (none at 
pres ent)^ 



VII-VIII Audio-print (e.g., Audi/Pointer®) 



^he decision to use these categories of communication media depends on 
data additional to what is provided in the curriculum analysis. Therefore, 
these categories are not included among the communication media suggested in 
question 29, Decision Table B. The combination of motion and still pictures 
in one system, which these categories represent, has an advantage mainly in 
the cost of med^a Software over motion pictures alone. Scenes involving no 
motion amount to nearly half the length of the average instructional film. 
Nor do most motion pictures require as many as the standard 2-4 frames per 
second for smoothness and realism. Thus, for cost and practicality, these 
multimedia have much to recommend them. ■ 

No example of this category is currently in use, although the Plato IV 
system is being developed at the University of Illinois, and Telestrator 
Industries of Chicago has built a variety of electronic devices that will 
perform this and other multimedia functions. Until 3uch models are opera- 
tional, the instructional designer should substitute class IV media or 
multimedia category IV-V when the CAQ designates this category. 



PART III 



Questions 30 through 33 attempt to establish the level of visual or audlal fidelity 
required if the presentation were to be made bj ^x^ommunication media instead of 
face-to-face. At ihis point, the user is concernaa only with presentations to the 
student, not with the student *s response. 

All media classes except VII (audio) and VIII (print) have some visual compo- 
nent (although the print media class is visible, it is not considered visual in this 
context); therefcjre, questions 30, 31, and 32 should be answered if x(29) = any value 
except VII or VIIL 

Question 33, concerning audio fidelity, applies only to the media classes and 
multimedia categories with sound, and thus should be answered only if x(29) = a 
value other than-IV, V, VI, VIII, or V-VI. 

Because soras: instructional designers may not be familiar with the standard 
measures of technical quality in such matters as gray scale and definition, some 
examples of each level are given, based on media that are commonly encountered. 
The decisions required are fairly gross; it usually will not make much difference if 
the user chooses a category that is a step too high or too low. 

The final version of the CAQ will probably provide a set of criteria to assist the 
user in each of these decisions, similar to those provided for the decision tables in 
question 29. At present, the options are merely described by example. 



30. 



31. 



Answer for all 
events for* 
u)hioh x(29) =_ 

I, II, III, IV, V, 
VI, II-III,V-VI 



I,II,III,IV,V, 
VI,II-III,V"VI 



Question 

What level of color fidelity is required 
of the visual presentation medium? 

Black and white (monochrome) pictures 
will suffice. 

Color will be used for coding or em- 
phasis, with no need for realism. 

Full color, with an approximation of 
realism (e.g., quality of color as 
seen in the average home TV receiver 
or as reproduced by the less expen- 
sive helical scan VTR recorders) . 

Pull and realistic color . 

What level of gray-scale fidelity is 
required of the visual presentation 
medium? 

Choose only if : 

x(30) = 1. Two tones, simple black 
and white , 

x(30) = 1 or 2. Three to six tones 
(typical of poster art, average or 



Code 



x(30) 
x(30) 



= 1 



= 1 



x(30) 
x(30) 



x(31) = 1 
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AnsiiSsr for - all 
evemis for 
vM:^: i x(29) = 



62. I,tI,III,IV,V, 
VI ,11-111, V-VI 



33. \I»II,III,VII, 
VXI-VIII, 
II^III 



Qu e^^lon Code 
be low- aver age honaH reception :of 

broadcastt::television) • x(31) = 2 

x(30) U 2, 3, or 4. A grsy scale 
of 7 to 12 tones (typical of above- 
aver a g e bxo ad c a s:t:: t e 1 e v 1 si o n r e c ep - 

tion, avexrage 16— mm or 8— mm film) . x(31) = 3 

x(30) =• 1, 2, 3, or 4. Need to dis- 
criminate between 13 or more tones 
(characteristic of photographic prints 
made from negatives of wide latitude; 
details are visible aimultane:ously in 
highlight and shadow areas) • x(31) =« 4 

What level of definition is required of 
the visual presentation medium? 

300 *'TV lines" or less. This is the 
level of definition associated with 
either color or black and white TV 

broadcast on an average home receiver. x(32) - 1 

Between 300 and 600 TV lines. This is 
the definition expected of a good closed- . 
circuit TV system, video (not R-F) dis- 
tribution, and display on high-quality 
monitors (not receivers) . Also charac- 
teristic of the quality of control-room 
broadcast TV, 2-in . broadcast standard 
video tape, or black and white 8-mm film. x(32) = 2 

Between 600 and 900 TV lines. This is • 

the definition usually associated with 

8-i3im color film or a high-def inition 

(10 WHz bandwidth) video system, x(32) ^ 3 

Over 900 TV lines. This is character- 
istic of Super-8 color film, 16-mm 
film, either black and white or color, 
.or a very high-definition, closed- 
circuit TV system with 18 MHz band- 
width distribution and over 1000 
scanning lines . 

x(32) = 4 

What level of audio fidelity is required? . 

Voice-quality audio is sufficient; no 

need for music or realistic sound, x(33) = 1 

Audio system should be capable of good 

music reproduction, e.g. , the quality 

of a typical AM radio broadcast. x(33) = 2 

High-fidelity, capable of realistic 
sound, e.g., typical FM radio 

reception. x(33) = 3 

Very-high-fidelity, capable of excel- 
lent sound realism^ e.g. , a' $2000 

stereo system, x(33) = 4 



IV. USE OF THE CAQ TO ANALYZE A COURSE IN 
BEGINNING PHOTOGRAPHY 



This ft^ection looks over the shoulder of a hypothetical instructional designer as 
he analyzes two learning events in a course on beginning photography by means of 
the CAQ. The course is relatively short, containing only 30 learning events; it could 
be completed by the average student in about 10 instructional days.^^ Completion 
of the CAQ for all learning events in the course is shown by the fiUed-in CAQ 
response form, Fig. 2. Each line in column j represents a learning event in the 
course, numbered according to its sequence in the topic, noted in column k. The 
columns denote CAQ question numbers. 

As is evident from the text of the CAQ in Sec. Ill, not all questions pertain to 
each learning event. The learning events in this course encountered as few as 18 and 
as many as 29 of the 33 questions. Here we follow two representative learning events 
through the analysis. The reader may want to refer back to the text of the CAQ and 
to Fig. 2 as the discussion proceeds. 



LEARNING EVENT 1-1 

The instructional designer has described learning event 1-1 (i.e., the first learn- 
ing event in the first topic) as follows: 

An introductory presentation covering a general orientation to photogra- 
phy as a career field, the study methods that will be used in the course, and 
an outline of overall course content. Students will not be examined (formally 
evaluated) for mastery of this material. 

The first eight questions (j, k, 1-6) are to be answered for all learning events. In 

"For a detailed description of this course, see Robert L. Petruschell and Polly Carpenter, MODI A 
Applied in the Design and Cost Analysis of an Innovative Air Force Course, The Rand Corporation, 
R-1021-PR, November 1972, 



21 



cn 








CSI 


^4 


N 




CS4 


CSJ 


CSJ 


C>4 


CSJ 


CM 


CM 


CM 


CSi 


CM 


CM 




CM 


CM 


CsJ 


CM 


CM 


CSJ 




c^ 


CM 


CM 


CM 


r> 








CM 






CSi 


>- 


CM 


CS! 


CM 




Csj 


CM 


CS4 








CSI 


CM 


CM 


>«> 




CM 


CSJ 


CSI 


CM 








n 








cn 


CO 












>«» 




CM 




to 






CS) 


CO 


CO 


«^ 






CO 


CO 


CO 


CM 


CM 






O 
n 






























































CN 


W 






N 


N 


w 




N 






U 








X 




N 






ft 








^ 








N 






CO 


o 


o 


o 


o 


o 


o 


o 








CSA 


rs 




VfS 


Cv 
v# 




VM 




nJ 


cs 




vSJ 


sT 






CO 






o 












o 








o 


















sj 




























>o 








o 






































rv 


















O 


o 




v.* 


o 








o 


O 


o 




o 


o 




o 






O 












Cs 


<s 


SJ 


rs 

VJ 


rs 








o 


o 


o 


o 


o 




o 


o 




O 


o 


o 


o 


o 


o 


o 


O 


O 


O 


o 


O 




o 


o 


o 


o 


o 


O 


o 


CO 
CN 






vO 


so 






t^ 


on 


CSJ 




vp 


\o 




NO 


GO 


CO 




CO 

«r 


CO 




sD 
CO 


$ 


00 




o 




O 
so 


o 
>o 


M) 


so 


<N 
CN 








































o 




>•* 








o 


O 




o 


































































o 

CN 




















































o 






o 




O 












o 


m 


o 


O 
cn 




4n 

CSJ 


to 

CM 






CO 


CO 


o 




o 

CSJ 


CSi 










o 

CM 




m 




o 


O 
«^ 


aa 






O 








O 










o 














O 


O 










vn 




in 


















































































>^ 


















































12 










































































































CM 


















cn 
















o 




























o 












o 


o 


o 


CN 








o 








CM 








CM 






CM 


o 


CM 
















CM 










eg 
































V* 
































o 




























































O 


M 






O 








o 










o 








O 








O 










0 












O 








o 


o 


o 


o 










O 








o 








0 














o 








CO 




























































o 


rs. 
















o 














O 




o 


















o 






o 


V5 












CsJ 




















CM 


c> 








*^ 


<^ 


OS 














OS 




O 


o 




o 


o 


O 


o 


o 


o 


o 




o 


O 




to 


O 


o 


o 


O 


o 


o 


o 


o 


o 


o 


o 


o 


O 


o 


o 




o 






o 




o 


o 






o 




o 


*^ 




O 






o 


o 


o 








o 


o 




o 




o 






o 


o 


O 






o 
























o 


























CN 


o 




o 




o 




o 








o 




o 


o 


O 






o 


o 












o 














tn 


IZO 


CSI 


}e 


o 


OZI 


o 

CO 




CO 


S0 


o 

CO 


b 

CO 




o 

sO 


o 

tM 




to 




o 
to 


CO 










o 

CO 


to 




o 

?0 




o 

CO 














M 








CM 


CO 






^« 


N 




Cn 




CM 


c^ 










N 


CO 






















N 






CO 


CO 


«o 


CO 


CO 


CO 


CO 


CO 


CO 



























j and k, respectively, topic numbers are assigned and learning events are numbered 
sequentially within their topics. 

Question 1 asks the designer to estimate the number of minutes required for the 
average student in the expected population to achieve the objectives of the learning 
event, given conventional methods and no time spent waiting for facilities or equip- 
ment to become available. For this learning event, the designer visualizes a passive 
group attending a classroom lecture and decides it would require 35 min; so he 
marks 35 on the response form; thus, x(l) = 35, There is no waiting time in this 
particular event, of course, since it involves no student practice with facilities or 
equipment. 

Question 2 asks the designer to estimate any additional time re;^uired for 
housekeeping activities. For the sake of simplicity, housekeeping in this course is 
assumed to be included in the time estimated in question 1 for all events; therefore, 
the answer to question 2 is always 0. In the real world, a separate estimate of time 
would ordinarily be made in answer to question 2. Actually, since the first learning 
event here is only a presentation and does not require the use of materials or 
equipment, no housekeeping time is required anyway. 

Question 3 seeks to determine the complexity of the learning event by asking 
whether mastery of the subject matter requires the integration of several simpler 
skills or concepts. In this example, the answer is *'No'* and x(3) = 0. Question 4 elicits 
further information on the difficulty of the subject matter by asking whether the 
average student in the learner population would find it relatively easy or relatively 
difficult to master. Here, the designer codes the question as x(4) = 0, indicating that 
he considers it easy. 

Question 5 determines whether this learning event is a basic part of the instruc- 
tional sequence or whether it is designed for a special subset of students, such as 
those who lack necessary prerequisites or those who have already mastered the 
subject matter. In this course, all learning events are intended for all students, so 
the answer is always 0. 

Question 6— a key question— asks that the learning event be assigned a number 
from 1 to 12 identifying the particular type and category of instruction it represents. 
Since learning event 1-1 requires no student performance, it is clearly Type I in- 
struction (see Table 1); because it requires no special facilities or equipment, it is 
coded category 1. 

Many of the following questions are answered or not answered according to the • 
instructional category number, so the designer scans the list in the left column of 
the CAQ, looking for "1." He finds it in question 19, which asks for an estimate of 
the time required only for the actual presentation or demonstration in the event. 
For this learning event, the designer enters 25 min, indicating that he expects 10 
of the 35 min estimated in question 1 to be devoted to introductory remarks, ques- 
tions by the students, etc. This question seeks to determine the duration of the 
presentation, so that if a communication medium is to be used, the amount of media 
software required can be estimated. 

As question 20 does not apply to category-1 instruction, the designer proceeds 
to question 21, which begins a set of questions to be answered for all learning events. 
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Question 21 asksnvhether this is to be considered a scheduled or unscheduled 
learning event. The designer decides x(21) = 1, indicating that the event is sche- 
duled in time and place. Question 22 asks whether students will be formally exam- 
ined on the material. Because this learning event is an introductory presentation, 
the designer enters x(22) = 0. That indicates that the objectives of this first event 
are intended to help the learners master the material in the second learning event, 
which will be formally evaluated.*^ Question 22 is important in determining the 
requirements of the evaluation sessions that cover this portion of the course. 

Question 23 relates to the possible use of communication media for the presen- 
tation, and asks how frequently its subject matter would ordinarily be revised. 
Because learning event 1-1 relates to the entire course content, revisions would 
occur fairly frequently. The designer estimates 12 months between revisions, hence. 
x(23) = 12. 

Question 24 determines if the event contains classified information. The course 
on basic photography contains no such information, so the entire column is coded 
0. Question 25 asks whether this event marks the end of a major course section; since 
it does not, x(25) = 0. This information is required to schedule any review or 
examination sessions. 

Questions 26 and 27 are to be answered only if x(25) = 1; they ask for estimates 
of the number of [wash-backs] or dropouts expected. Since x(25) = 0, these questions 
do not apply. 

Question 28 applies to all learning events. It asks for an estimate of the percent- 
age of students that are apt to need remedial instruction before being able to master 
the subject matter in the learning event. Since this event was previously classed as 
easy (x(4) = 0), and since it is only introductory, no remedial instruction is expected, 
and x(28) = 0. 

The designer then moves on to Part II of the CAQ, which concerns selection of 
the most appropriate class of communication media. (The decision whether or not 
to actually use communication media will be made later in setting the strategy.) If 
the strategy specifies face-to-face presentation, for example, this information will 
help select the appropriate instructional aids. 

Question 29 is actually a series of questions, presented as Decision Tables A and 
B (see pp. 17 and 18). Beginning with Decision Table A, the designer answers ques- 
tions kbout the presentational needs of the learning event, referring to the criteria 
in the footnotes. The pattern of Yes-No answers on the table that corresponds to his 
pattern of answers leads to the designation of the simplest appropriate communica- 
tion media class. 

In the present case, the designer's pattern of answers corresponds to the pattern 
that directs him to Decision Table B, indicating that the subject matter in this 
learning event falls in the conceptual rather than the perceptual domain. 

Now at Decision Table B, the designer considers the first question about the 
desirability of presentation by audial means rather than print. Referring to the 
criteria in the footnotes, he decides that the first criterion applies: many of his 

" Such objectives are often termed cwc6/m^ o^i/cc^ri;€5. 
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students would understand the information better if they heard it rather than read 
it. He records a 'T." 

The next question asks about the desirability of print or other visuals to accom- 
pany the audial presentation. Since the designer is thinking of using a list of lesson 
topics for the course, a list of careers in which photography is used as a professional 
tool, and perhaps seme pictures of aerial photographers at work (to help the student 
form an image of himself as an Air Force photographer), he decides that„ visual 
means would be desirable^^ and enters a "Y." ' 

The next question asks whether pictures would be desirable in addition to print 
and black-white graphics. Thinking of the photos of aerial photographers, he an- 
swers "Y." 

The fourth question asks about the desirability of motion in the visuals. Our 
designer can think of no reason for motion and enters ''N." Similarly, the last 
question, about the desirability of semimotion, elicits a "N"; he can think of no 
purpose for '^pointing" or "buildup" (writing or drawing), the two basic kinds of 
semimotion. His pattern of three Y's followed by two N's corresponds to the pattern 
that designates class II as the simplest appropriate media class to use for this 
learning event. He enters a II for x(29) on the response form. 

Then, he begins Part III of the CAQ. In response to question 30, "What level of 
color fidelity is required?" the planner decides that black and white will suffice, and 
ansv/ers x(30) == 1. In response to question 31, "What level of gray-seale fidelity is 
required?" the planner decides that he needs the tonal quality of good television or 
film presentation and answers x(3l) = 3. He might have needed only two tones, 
black and white, were it not for the pictures of aerial photographers at work. 

Question 32 asks for the necessary level of definition. The designer believes that 
the average home TV reception is not sharp enough; better:definition is required to 
show ground and cloud details in the photograph, to make it dear that the photogra- 
pher is in an airplane. He thus marks x(32) = 3, the level of definition of a high- 
quality television system. Finally, in answer to the last question, which asks for 
audio fidelity requirements, the planner enters x(33) = 2— the level of quality of 
good AM radio reception. That is suflScient because the only important sound will 
be the instructor's voice. 



LEARNING EVENT 4-5 

As the first learning event followed through the CAQ. was Type I instruction, 
category 1, we did not encounter all the questions in the questionnaire. To illustrate 
how questions that apply to other instructional categories might be answered, we 
chose learning event 4-5 (the fourth learning event of the fifth topic) to take through 
Part I of the CAQ (answering the other parts of the CAQ will not .vary as much with 

The affective objective— the student^s formation of a positive future self-image— could have led the 
designer to believe that visual means v/ere required, and to select a different-naie in Decision Table A 
In either case, however, communication media class II would have been indicated. 
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the type of instruction). The objectives of learning event 4-5 were described as 
follows: 

Each student takes two pictures and develops the exposed cut lilm. Here he 
will get a chance to put into practice the process of developing film that has 
been demonstrated and explained to him. Given a familiar 4x5 press 
camera and familiar film stock, after mastery of this event on an exam he 
will be able to properly expose and correctly develop a negative under simi- 
lar conditions to the satisfaction of his instructor. 

Learning event 4-5 is estimated to require 60 min to complete and to be both 
relatively complex and relatively difficult to master. In question 6 it is classified as 
category 9 instruction because it is the pure performance'of individual skills that 
requires special equipment or facilities. 

The next question encountered for a category 9 learning event is question 7, 
which asks whether speed of performance is an objective of the event: For this event, 
it is not; hence x(7) — 0. 

Question 10 asks if an instructor must be present to provide the student with 
knowledge of correct responses, and the designer answers 'Tes." That automatically 
determines the answer to question 11, which asks what level of supervision is 
necessary to prevent injury to either the student or the equipment. The designer 
realizes that he must enter (supervision by an instructor) if he has entered "1" 
for the previous question, because if the instructor must be present to let the student 
know how he is doing, he will be available for safety supervision as well. 

Question 12 asks what special area is required. Since the objectives say nothing 
about where the two photographs are to be taken, it is a minor consideration. The 
designer, thinking only of the place needed to develop the photographs, codes x(12) 
= 4, a small laboratory — in this case a darkroom. 

Question 13 asks if student mastery of the skill involved can be diagnosed by 
having the student make selected responses. In this case the answer is *'No," so x(13) 
= 0. Questions 14 and 15 are answered only if x(13) ^ 0. Question 14 asks the nature 
of the constructed response required. Since the student is actually producing some- 
thing, rather than using verbal or graphic means to express a thought or solve a 
problem, the planner codes x(14} ^ 2, a student product. 

Question 15 asks what means would be most appropriate for recording students' 
constructed responses. A number of specific communication media are listed that 
can (1) express and record information by students (e.g,, paper and pencil); (2) keep 
a record of student performance that would otherwise be lost (e.g., a recording of the 
pronunciation of foreign words); or (3) actually constitute a student product (e.g., a 
student film). A final category is available if the product is 'to be made of materials 
other than communication media (e.g., assembly of an electronic circuit). In this 

Note that this criterion for an acceptable level of skill— "satisfactiion of the instructor'*— is not an 
empirical measure. That reflects a problem encountered in specifying most constructed responses. '*Cor- 
rectness" has nothing to do with "artistry*' in a photograph, for example. The best way to determine the 
attainment of artistic quality is to set up a board of review and average out the subjective judgments of 
a number of experts. Since that is impractical in the present course, the instructor's judgment has to 
suffice as the criterion. 
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learning event, still photography can serve at least two purposes, and because the 
student must expose and develop a photographic negative, the designer answers 
xdS) ^ 4. 

Question 16 asks whether special equipment is required for the instructional 
presentation. Equipment that might be necessary if a prese^itation were to be trans- 
mitted or recorded is not considered. In this case, cameras and darkroom equipment 
are needed for the instruction, so the answer is "Yes," x{16) = 1. Question 17, to be 
answered only if x(16) = 1, asks if the special equipment is already on hand. In this 
case, we assume that the designer has all the necessary facilities and equipment, so 
x(17) - 1, "Yes." 

Question 19 asks for an estimate of the time required, in this case, to give 
problems and directions. The designer estimates 5 min, so x(19) == 5. The last 
question in Part I that pertains to instructional category 9 is question 20. It asks 
whether students will require special materials. Thinking of the raw jBlm that will 
be exposed, the designer replies "Yes/' and x(20) = 1. 
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V. TESTING THE CAQ 



The task that lies ahead is to test the CAQ in use by actual instructional 
designers to determine if it elicits the information it was designed to obtain. If it does 
not, the phrasing of the questions or the format of the questionnaire will have to be 
modified. Empirical evaluation will inform us how long it takes an instructional 
designer to analyze a given number of learning events. We can also get a good idea 
of how much assistance or insight the designer gains merely by using this tool in 
its manual form. We have not attempted to design in detail a field evaluation 
program. These comments are intended only to suggest some useful components of 
such a program. 

Although the CAQ is designed to be used by an instructional designer without 
assistance^ testing of the CAQ must be done in an interview format, by someone 
familiar with the questionnaire and the intent of each question. Failure to elicit the 
desired information can thus be immediately detected and notes made to guide 
revision. 

The CAQ has already benefited from one such test. The team of three persons 
who had worked on the questionnaire interviewed an instructional designer for 2 
or 3 days. Each question was asked verbally and the respondent's answer and any 
discussion were recorded. First, an existing course was analyzed and a number of 
its learning events were taken through the CAQ. Then, the designer was asked to 
give his imagination free rein and think of the course as he would like to teach it, 
thus making it an entirely new course. 

This procedure, although crude, proved extremely valuable. As a result, the 
questionnaire was thoroughly revised; questions were added, subtracted, and rewrit- 
ten in accordance with our findings. 

Future testing, however. Will have to be more carefully designed and controlled. 
More than one designer should be interviewed, preferably designers planning differ- 
ent kinds of courses and designers with various educational backgrounds and vary- 
ing amounts of experience. Interest in instructional innovation would be another 
important variable to have represented in the testing, as well as prior experience 
in designing instruction using comrriunication media. . 

The questionnaire is designed to be applicable to any kind of instruction at any 
educational level. Thus, the CAQ should be tested with a variety of courses, to 
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determine if the questions are universally applicable. Since the evaluation would 
not attempt quantitative estimates, the size of the sample would not be a critical 
factor. 

Table 3 lists some of the important variables that should be considered and 
some of the values pertaining to them. Although the total number of combinations 
of all the values for all the variables listed could amount to several hundred, it 
should not be necessary to select more than 10 courses and their designers to 
satisfactorily validate the CAQ. If necessary, the process could be repeated with 
ensuing revisions. To reduce the effects of inadvertent bias in the interviewer, at 
least two interviewers should be used. 

A possible procedure might be, first, to test the directions and format of the 
questionnaire by asking the designer to read and answer silently, say, the first five 
or six questions. Any difficulties encountered could be discussed. Then, the remain- 
der of the questionnaire could be answered and discussed orally. 



Table 3 

VARIABLES TO BE INCLUDED IN EMPIRICAL TESTING OF CAQ 



Variable 


Values 


Courses 


Educational level 


Elementary; secondary; technical 
training; post-secondary 


Course level 


Elementary or advanced; 
basic or specialized 


Type of instruction 


Theory; practice 


Inception 


New course; redesign of existing 
course 


Instructional Designers 


Sex 


Male ; female 


Formal training in instructional 
design 


None; some; a lot 


Educational attainment 


Secondary school only; college; 
graduate study 


Familiarity with the use of 
coTTununication media in 
instruction 


None ; conventional instructional 
aids only; print media; audial, 
visual, or audio-visual media 


Attitude toward innovation in 
instruction 


Conservative; liberal 
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Since in Part I any one learning event would be subjected only to the questions 
that apply to its particular category of instruction, difFerent kinds of learning events 
should be analyzed until each question has been encountered at least once. Since the 
purpose is fx) test the questions, not to analyze a curriculum, the testing procedure 
could skim through a course, analyzing those learning events that would be likely 
to involve questions that had not yet been asked. If it became obvious that a course 
would not involve several categories of instruction, and hence that not all questions 
would be encountered, special courses could be sought to test out the remaiining 
questions. , 

For each course, approximately 20 learning events should be analyzed. Some 
questions may apply to all of them; others to only a few. One interview may take 
as long as two or three hours. Although the time required for the interview will not 
be a reliable guide to the time required for the solitary instr'!'^.tional designer to go 
through the CAQ, it may be possible, at least during the later interviews, to estimate 
that time. Courses should be chosen that are actually being designed or redesigned. 
Preliminaries such as the formation of the curriculum, the ordering of learning 
objectives, and the subdivision of the course into learning events, must have been 
done before the evaluation interview. Where there is a choice, it would be wise to 
select designers who are articulate, or at least talkative, in the hope of obtaining a 
better resolution of any problems that might arise in interpreting the interview 
results. 
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GLOSSARY 



Affective domain: Subject matter pertaining to feelings, emotions, attitudes. 

Cognitive domain: Subject matter pertaining to the recognition or recall of 
knowledge and the development of intellectual capabilities and skills. 

Communication medium: A means of communicating that is self-contained, 
i.e., it can convey a message without the message sender's being at the point of 
reception.. Audiovisual materials, such as flip charts, mock-ups, and "Vu-graphs,*' 
are communication aids, not communication media. There are eight classes of com- 
munication media, according to the author*s taxonomy (see inside front cover). 

Constructed response: A student performance or product, e.g., speaking/ 
writing, drawing, gesturing, using a tool/operating a machine, making something. 
Compare selected response. 

Conventional instruction: Classroom and laboratory group methods in 
which lecture/oral quiz, guided discussion, drill, and the use of texts and workbooks 
predominate. Includes demonstrations by an instructor. 

Discussion: (1) A fairly informal session set aside after a segment of b^sic 
instruction to answer students' questions or to stimulate student discussion of the 
subject matter covered. Not considered a separate learning event, but taken into 
account in the MODIA process on the basis of stated strategy. (2) A segment of 
instruction to teach students to converse and interact in a group. Considered a 
learning event of Type II instruction (team skills). 

Drill: An activity intended to help the student learn a relatively simple skill 
by repeated, fairly mechanical response to a stimulus. The stimulus is usually 
external, as in typing from a text or repeating foreign wordr. after a model to practice 
pronunciation. The student provides his own stimuli in most independent study. 
Compare practice. 

Evaluation: The measurement or assessment of student achievement; may 
be formal or informal. Common means are the quiz, performance test, final examina- 
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tion. Used to measure students' progress and sometimes to regulate their advance- 
ment in a Course of study. Evaluation segments are not considered separate learning 
events in the curriculum analysis. The MODIA process automatically accounts for 
them on the basis of stated instructional strategy. 

FoUow-me instruction: The step-by-step demonstration of procedures, 
which students copy, or student practice following step-by-step directions. May occur 
in either Type II or Type III instruction; applies only to individual skills. 

Formal evaluation: An examination. 

Housekeeping: Giving assignments or announcements, classroom manage- 
ment, cleanup, etc. The time required for housekeeping will often depend on the 
activities carried out during the class session; it is usually greatest with Type III 
instruction. Not considered a separate learning event in the curriculum analysis. 

Independent study: Study by individual students or groups of students at a 
time of their own choosing, at home or at school, with no instructor or monitor 
present. Synonym for unscheduled instruction. 

Instructional aid: Either a communication aid used by an instructor (e.g., flip 
charts) or a learner's aid, such as an actual or simulated object being studied. 

Learning event: A basic segment of homogeneous instruction. Does not in- 
clude question-and-answer sessions, discussions (in the first sense), scoring of desk- 
work or homework in class, examinations, housekeeping, coffee breaks, or transpor- 
tation. 

Materials: See media software. 

Media software: Recorded program materials, e.g., films, tapes, books, discs, 
etc., containing mep?.^»ges; Used as instructional or communication aids. 

Performance: Student activity in Type II or Type III instruction, in which 
the student learns how to do something, such as a skill, not simply absorbs ideas or 
information. Student responses in Type I instruction (answering questions, filling in 
blanks in a workbook, etc.) are not considered performance. 

Performance model: Demonstration of a skill that a learner is expected to 
master. 

Practice: Student performance in learning relatively complex skills, as dis- 
tinguished from drill, performance in learning relatively simple skills. Practice is 
less repetitive and automatic than drill; a student may practice a complex skill, such 
as troubleshooting or writing a business letter, only a few times during the course 
of study and may have, a slightly different assignment each time. As regards prac- 
tice, the presentation of stimuli is less important than for drill, but the demonstra- 
tion of the skills to be learned is more important. 
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Presentation: Any communication to the student. Includes transmitting 
facts or concepts, demonstrations, giving directions, supplying stimuli for drill, 
describing problems to be solved. 

Print: A way of representing information using symbols such as alphanu- 
meric characters, pictographs, logograms, ideograms,' hieroglyphics, and handwrit- 
ing. Does not include line graphics or pictures. 

Psychomotor skills: Skills requiring muscular movements, usually 
manipulative, including hand-eye coordination, pronunciation, handwriting, and 
using tools. Because most such skills require cognitive support (are more than 
automatic motor responses), we use the term psychomotor to refer to all motor skills. 

Question-and-answer session: A peiiod for the informal answering of stu- 
dents' questions or discussion (is a synonym for discussion in the first sense). Does 
not include questions answered during the course of instruction. Not considered a 
separate learning event in the curriculum analysis; automatically taken into ac- 
count in the MODIA process on the basis; of stated strategy. 

Remedial instruction: Instruction an tended to assist students who have 
failed to master a particular learning event. Does not include sessions to review or 
make up work. 

Review: An abbreviated version of earlier instruction intended to refresh the 
learner's memory or skill in preparation for an examination. Such sessions are not 
considered separate learning events in the curriculum analysis; the MODIA process 
automatically accounts for them on the basis of stated strategy. 

Scoring deskwork: Scheduled instructional time set aside for evaluation of 
a student's work (usually of the selected response type) by the student himself or 
by another student. Such sessions are not considered learning events in the cur- 
riculum analysis. MODIA automatically aJ^counts for them on the basis of stated 
strategy. 

Selected response: A means of eyahiasbion in which the student selects one 
answer from a choice of two or more, or places a list of items in the correct order. 
Multiple-choice and true-false questions, anclinatching and ordering tests are com- 
mon forms. Compare constructed respon^se^ 

Semimotion: ''Buildup" or "pointing'Vor both. Buildup is the accretion of 
lines or symbols, as in writing or drawing; ipoii^tlting is the indication of points, areas, 
or routes on a picture or line graphic by thermotion of a pointer, spot, or symbol. 
'Top-in" of labels, lines, etc., is not semimotion and can be done by a sequence of 
two or more still pictures. 

Special equipment: Equipment or tools that students must use other than 
the traditional paper and pencil, drawing instruments, slide rules, or other small* 



implementsv Such equipmBnt (1) is so expensive audr fragile (such as precision mea- 
suring insfemments) or dangerous (such as corrosi^iacids) that student use must be 
supervised;: or (2) requires^ispecial facilities to be used. Projectors, playback devices, 
and the life, which are used for communicating with the student, are not included 
unless they themselves are the subject of study. 

Special facilities: Areas that ars different from a conventional classroom, 
such as workshops, laboratories, hangars, playing fields, and gymnasiums. 

Team: Two or more persons who must work together, e.g., participants in a 
two-handed game or people using a transit and chain to determine the placement 
of a survey marker. People working individually in a class do not constitute a team. 

Topic: A coherent set of ideas or skills that the student masters as a unit of 
subject matter in a course and that can serve as the subject of a formal evaluation. 
Can span several class sessions or can be part of a single cleiss session. 

Unscheduled instruction: See independent study. 

Wash back: To cause a student to repeat a segment of instruction, usually by 
placing him in a group of less advanced students. 
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